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ABSTRACT 

The General Accounting Office (GAO) reviewed the performance 
measurement system of the Department of Labor's (DOL's) Veterans Employment 
and Training Service (VETS), which administers programs and activities 
designed to help veterans obtain employment and training assistance. The GAO 
review focused on the following items: VETS' proposed performance measures; 

(2) the data sources VETS proposes to use in its new system; and (3) any 
other measurement issues that might affect the comparability of states' 
performance data. The review was based on interviews and discussions with 
more than 45 officials in 15 states, interviews with VETS officials, and a 
review of relevant documents. The review established that although VETS' 
proposed performance measures improve performance accountability over the 
current system, certain aspects of the new system raise concerns that VETS 
should address. It is recommended that VETS establish a performance measure 
that more effectively gauges the success of its staff-assisted services and 
that it clearly define its target population. The GAO further recommends that 
VETS eliminate the measure related to the number of federal contractor jobs 
listed. The following items are appended: a comparison of performance 
measures used by the VETS, Employment Service, and Workforce Investment Act 
programs; states' entered-employment rates for veterans in program year 1999; 
and comments from the DOL. (MN) 
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Accountability * Integrity * Reliability 



United States General Accounting Office 
Washington, DC 20548 



May 15, 2001 

The Honorable Christopher H. Smith 
Chairman, Committee on Veterans’ Affairs 
House of Representatives 

Dear Mr. Chairman: 

The Department of Labor’s Veterans’ Employment and Training Service 
(VETS) administers programs and activities designed to help veterans 
obtain employment and training assistance. Recently, policymakers have 
focused increasing attention on VETS and its programs, advocating 
changes to the structure and administration of the program and in the way 
it assesses program performance. For example, in 1999, the Congressional 
Commission on Servicememb ers and Veterans Transition Assistance 
recommended that the Congress establish effective outcome measures for 
VETS. In addition, legislation was introduced during the 106th Congress 
that would restructure the program and require a new, comprehensive 
performance accountability system. 1 This focus on reform corqes at a time 
when other federally funded employment and training programs are 
changing the way they provide services and measure performance. For 
example, the Workforce Investment Act (WIA), which provides 
employment and training assistance for youth, adults, and dislocated 
workers through one-stop centers, recently established outcome measures 
that are similar to those that VETS now proposes. With these new 
measures, VETS has the opportunity to bring its performance management 
system in line with those of other key employment and training programs. 

Because of the Committee’s interest in improving the way employment 
services are provided to veterans, you asked us to review VETS’ 
performance measurement system. Specifically, you asked us to review (1) 
VETS’ proposed performance measures, including possible areas of 
concern regarding the measures; (2) the data source VETS proposes to use 
in the new system; and (3) any other measurement issues that may affect 
the comparability of states’ performance data. Our review is based on 



1 H.R. 4765, 106th Cong. (2000). This bill also provides for incentive grants based on state 
performance in carrying out veterans’ employment, training, and placement services. 
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Results in Brief 



interviews and discussions we had with over 45 officials in 15 states, 2 
interviews with VETS officials, and a review of relevant documents, 
including VETS’ 2000-2005 draft strategic plan as of March 2001. We 
conducted our review from October 2000 to April 2001 in accordance with 
generally accepted government auditing standards. 



VETS’ proposed performance measures improve performance ' , 

accountability over the current system, but certain aspects of the new 
measures raise concerns that VETS should address. The measures 
currently in place focus largely on the process — that is, the number of 
services provided to veterans. Under the proposed system, more focus is 
given to measuring the outcomes veterans achieve from the program. In 
addition, under the proposed system, VETS no longer requires states tp 
compare services provided to veterans with those provided to nonveterans 
when establishing expected levels of performance. The proposed 
measures are also more closely aligned with those of WIA, making it easier 
for service providers to report on outcomes. While the proposed measures - 
improve the way in which VETS will measure program performance, a few 
areas of concern remain. VETS’ strategic plan suggests that states focus 
their efforts on providing staff-assisted services to veterans, including case 
management. Yet, none of the proposed measures specifically gauge the 
success of these services and may instead encourage staff to focus their 
efforts on assisting veterans who may more easily find jobs. In addition, 
VETS’ proposal includes one measure — the number of federal contractor 
jobs listed with local employment offices — that is not only process- 
oriented but also focuses on outcomes that are beyond the control of staff 
serving veterans. We are recommending that VETS establish a 
performance measure that more effectively gauges the success of these 
staff-assisted services and that VETS clearly define its target populations. 

In addition, we are recommending that VETS eliminate the measure 
related to the number of federal contractor jobs listed. 



2 We chose to interview officials from these 15 states because they were familiar with 
performance measurement issues. The states included: California, Colorado, Connecticut, 
Florida, Illinois, Indiana, Michigan, Missouri, Montana, New Jersey, Oregon, Pennsylvania, 
Tennessee, Texas, and Washington. Officials from these states participated in a workgroup 
that was convened by Labor’s Office of Workforce Security in January 2000. It was tasked 
with developing labor exchange performance measures, revising the data collection and 
reporting systems provided under the Wagner-Peyser Act, and developing procedures for 
establishing levels of performance for the delivery of labor exchange services. 
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VETS proposes that all states use a single data source — Unemployment 
Insurance (UI) wage records — to identify veterans who get jobs. Using 
these data will greatly improve the comparability and reliability of the new 
measures, because it will standardize the data states use to report their , 
performance. Currently, the data used by VETS are not comparable across j 
states, in part, because states use a variety of data sources for 1 i- 
performance-reporting purposes. In addition, we found evidence that 
using wage record data will help state staff better identify those veterans j 
who find work after receiving services. While using these data will / 

improve some aspects of data collection, the data present some 
challenges. States generally do not have access to wage records from other 
states and, therefore, should find ways to track individuals who receive 
services in one state but get a job in another. In addition, VETS’ programs 
should identify, if necessary, those veterans finding jobs in categories riot 
covered by wage records, such as self-employed workers and federal 
employees. 

As VETS finalizes its performance-reporting requirements, other issues 
that affect the comparability of states’ performance-related data should be 
considered. For example, states vary in whether they register and count, 
for performance reporting purposes, job seekers who use only self-service 
tools, such as internet-based job listings. In addition, states differ in how 
long a veteran remains registered with the state after seeking services. 
These differences in state registration procedures affect how states 
calculate the number of veterans who get jobs and make comparing 
performance across states less reliable. In order to have performance data 
that are comparable, we are recommending that VETS standardize the way 
states register and count veterans for reporting purposes. 



Background 



VETS administers national programs intended to (1) ensure that veterans 
receive priority in employment and training opportunities from the 
employment service; (2) assist veterans, reservists, and National Guard 
members in securing employment; and (3) protect veterans’ employment 
rights and benefits. The key elements of VETS’ services include enforcing 
veterans’ preference and reemployment rights and securing employment 
and training services. VETS’ programs are among those federal programs 
whose services have been affected by WIA and other legislative changes 
aimed at streamlining services and holding programs accountable for their 
results. 
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VETS’ Programs 



VETS carries out its responsibilities through a nationwide network that 
includes representation in each of the Department of Labor’s 10 regions 
and staff in each state. The Office of the Assistant Secretary for Veterans’ 
Employment and Training administers VETS’ activities through regional 
administrators and a VETS director in each state. These federally paid 
VETS staff are the link between VETS and the states’ employment service 
system, which is overseen by Labor’s Employment and Training 
Administration (ETA). VETS funds two primary veterans’ employment 
assistance grants to states — the Disabled Veterans’ Outreach Program 
(DVOP) and the Local Veterans’ Employment Representatives (LVER). , 
Fiscal year 2001 appropriation for VETS was about $183 million, including 
$81.6 million for DVOP specialists (DVOPS) and $77.2 million for LVER 
staff. These funds paid for 1,327 DVOP positions and 1,206 LVER positions. 

The roles of the DVOPS and LVERs have been separately defined in two 
statutes. LVERs were first authorized under the original GI bill (the 
Servicemen’s Readjustment Act of 1944) and DVOP specialists were 
authorized by the Veterans’ Rehabilitation and Education Amendments of 
1980. A key responsibility of a DVOP is to develop job and job training 
opportunities for veterans through contacts with employers, especially 
small- and medium-size private sector employers. LVERs are to' provide 
program oversight of local employment service offices to ensure that 
veterans receive maximum employment and training opportunities from 
the entire local office staff. In addition, DVOPS and LVERs traditionally 
have provided services that include 



• locating veterans who need services, 

• networking in the community for employment and training programs, 

• bringing together veterans looking for work and employers seeking to 
fill job openings, 

• making referrals to support services, and 

• providing case management for those veterans in need of more 
intensive services. 

Increasingly, however, veterans are accessing services on their own, 
through tools such as internet-based job listings or resume writing 
software. 

As part of the DVOP and LVER grant agreements, states must provide or 
ensure that veterans receive priority at every point where public 
employment and training services are available. The DVOP and LVER 
programs give priority to the needs of disabled veterans and veterans who 
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served during the Vietnam era. States’ employment service systems are 
expected to give priority to veterans over nonveterans. Generally, this 
means that local employment offices are to offer or provide all services to 
veterans before offering or providing those services to nonveterans.j To , 
monitor the states’ programs, VETS has been using a set of measures that • 
evaluates states’ performance in five dimensions: (1) veterans placed in « 
training, (2) those receiving counseling, (3) those receiving services, (4) , 
those entering employment, and (5) those obtaining federal contractor j 
jobs. These measures primarily count the number of services that veterans 
receive and compare the totals with similar services provided to 
nonveterans. To ensure priority service to veterans, VETS expects levels of 
performance for services provided to veterans to be higher than levels for 
nonveterans. For example, veterans and other eligibles must be placed in 
or obtain employment at a rate 15 percent higher than that achieved by 
nonveterans. (See table 1 for VETS’ specific performance standards.) 
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Table 1: VETS’ Current Performance Measures and Standards 


Measure and category of veteran 


Standard for veterans 
over nonveterans 
(Percent) 


Placed/obtained employment 


Veterans and other eligibles 


15 


Vietnam-era veterans 


20 


Disabled veterans 


25 


FCJL placements 8 


Vietnam-era veterans 


20 


Special disabled veterans 6 , 


25 


Number counseled 


Veterans and other eligibles 


15 


Vietnam-era veterans 


20 


Disabled veterans 


25 


Number placed in training 


Veterans and other eligibles 


15 


Vietnam-era veterans 


20 


Disabled veterans 


25 


Number receiving a reportable service 


Veterans and other eligibles 


15 


Vietnam-era veterans 


20 


Disabled veterans 


25 



“Veterans placed in jobs with federal contractors. 

b A special disabled veteran is (1) a veteran who is entitled to compensation (or who, but for 
the receipt of military retired pay, would be entitled to compensation) under laws 
administered by the Department of Veterans Affairs for a disability rated at 30 percent or 
more or (2) a person who was discharged or released from active duty because of a 
service-connected disability. 

Source: Veterans’ Employment and Training Service, Department of Labor. 

To report on performance, VETS currently relies on the Employment and 
Training Administration’s 9002 system to aggregate data reported by states 
on veterans and nonveterans who register with state Employment Services 
(ES) offices, track the services provided to them (such as counseling or 
job referral), and gather information on their employment outcomes. The 
9002 system also collects information such as the registrants’ employment 
status, level of education (e.g., high school, postsecondaiy 
degree/certificate), and basic demographic information, such as age and 
race. 
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Over the past several years, hie Congress has taken steps to streamline 
and integrate services provided by federally funded employment and 
training programs. WIA, which the Congress passed in 1998, requires 
states and localities to use a one-stop center structure to provide acjcess tq 
most employment and training services in a single location. WIA requires 
about 17 categories of programs, including VETS and ES programs, 'to i 
provide services through the one-stop center. However, because DyOP , 
and LVER staff can provide assistance only to veterans, and because their 
roles in one-stop centers are not specifically addressed in WIA, it is 
unclear how they will function with regard to one-stop centers. According 
to VETS officials, this lack of clarity has been addressed. Agreements 
made with each state on planned services to veterans now include 
provisions on how DVOPS and LVERs will be integrated into the one-stop 
delivery system. , ' 

In addition to changing the way services are provided, programs are now 
increasingly held accountable for their results. Through the Government 
Performance and Results Act of 1993 (GPRA), the Congress seeks to 
improve the efficiency, effectiveness, and public accountability of federal 
agencies as well as improve congressional decision making. GPRA does 
so, in part, by promoting a focus on what the program achieves rather than 
tracking program activities. GPRA outlines a series of steps in which 
agencies are required to identify their goals, measure performance, and 
report on the degree to which those goals were met. Executive branch 
agencies were required to submit the first of their strategic plans to the 
Office of Management and Budget and the Congress in September 1997. 
Although not required by GPRA, Labor’s component agencies, such as 
VETS, have prepared their own strategic and performance plans at the 
direction of the Secretary of Labor. 

To address the goals of GPRA and in response to recommendations by us 
and other groups, such as the Congressional Commission on 
Servicemembers and Veterans Transition Assistance, 3 VETS is currently 
developing a new system to measure the performance of its programs. 
Over the last several years, VETS conducted pilot programs in about eight 
states that tested some new performance measures and the use of new 



3 See Report of the Congressional Commission on Servicemembers and Veterans Transition 
Assistance, January 14, 1999, Congressional Commission on Servicemembers and Veterans 
Transition Assistance, Arlington, VA. 
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data to support these measures. VETS officials told us that they anticipate 
implementing their new performance measurement system in July 2001. 


VETS’ Proposed 
Measures Hold 
Promise, But Some 
Concerns Remain 

i 

t 


VETS’ proposed performance measures are a significant improvement 
over current measures, but certain aspects of these measures raise 
concerns that VETS may need to address. The proposed measures include 
an (1) entered-employment rate, (2) employment rate following staff- 
assisted services, (3) employment retention rate, and (4) increase in the 
number of federal contractor job openings listed. These measures are an 
improvement over current measures because they 

1 

• focus more on what the programs achieve and less on the number of 
services they provide, 

• no longer use the level of services provided to nonveterans as the 
standard for services that must be provided to veterans, 

• adjust expected state performance to economic conditions within 

states, and • 

• establish two measures that are already collected for WIA-funded 
services and proposed for ES. 

However, even with these improvements, the proposed measures continue 
to send a mixed message to staff about where to place their service 
priorities. In addition, the proposed measures include a redefined measure 
for tracking federal contractor job openings, but the measure is process- 
oriented and outside the scope of the work of DVOPS and LVERs. 


Proposed Measures Are An 
Improvement Over the 
Current Ones 


The proposed performance measures improve accountability because they 
place more emphasis on employment-related outcomes by eliminating 
process-oriented measures — measures that simply track services provided 
to veterans. Current process measures that VETS eliminated from the 
proposed performance system include the number of veterans referred to 
counseling, the number placed in training, and the number receiving 
certain other services, such as job referrals. As we noted in past reports, 
these process-oriented measures are activity- and volume-driven and focus 
efforts on the number of services provided, not on the outcomes veterans 
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achieve. 4 These measures offer states little incentive to provide services to 
those veterans who are only marginally prepared for work and who may 
need more intensive services requiring more staff time. The VETS’ 
proposal still includes one process-oriented measure that simply reflects 
the percentage increase in the number of federal contractor job openings 
listed with the public labor exchange but adds two outcome-oriented 
measures — job retention after 6 months and the employment rate i 
following staff-assisted services. 6 The VETS’ proposal also retains an I 
outcome measure that is in the current system — the entered-employment / 
rate. (See table 2.) 



4 See, Veterans’ Employment and Training Service: Focusing on Program Results to 
Impro ve Agen cy Performance ( G A O/T- H E H S-9 7- 1 29 , May 7, 1997) and Veterans’ 
Employment and Training Service: Better Planning Needed to Address Future Needs 
(GAO/T-HEHS-00-206, Sept. 27, 2000). 

6 VETS considers this measure as process-oriented “with an emphasis on outcomes.” 
However, for this report, we classified the measure as outcome-based because it reports an 
employment rate rather than only reporting a count of services. 

JEST COPY AVAILABLE 
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Table 2: VETS’ Current Performance Measures Compared With Proposed Measures 



Current measures 


Process- 

oriented 


Outcome- 

oriented 


Entered-employment rate: 

The percentage of all registered veterans who were placed in 
or obtained employment. 




X 


Number of veterans placed in training: 

A count of the veterans placed in training. 1 


X 




Number of veterans receiving counseling: 

A count of the veterans who received counseling services. 


X 




Number of veterans receiving some reportable service: 
A count of the veterans who received at least one reportable 
service. i 


X 




Federal contractor jobs filled by Vietnam and special 
disabled veterans: 

A count of the veterans who were placed in jobs listed on the 
federal contractor job list. 




X 


Proposed measures 


Process- 

oriented 


Outcome- 

oriented 


Entered-employment rate: 

The percentage of all registered veterans who were placed in 
or obtained employment. 




X 


Employment rate following receipt of staff-assisted 
services: 

The percentage of registered veterans who are employed after 
receiving some form of staff-assisted labor exchange services. 


' 


X 


Employment retention rate at 6 months: 

Of the veterans who had entered employment following 
registration, the percentage of those who continued to earn 
wages 6 months after entering employment. 




X 


Federal contractor job openings listed with the public labor 
exchange: 

The percentage increase in the number of federal contractor 
job openings listed annually with the public labor exchange 
from one program year to the next. 


X 





Source: Veterans’ Employment and Training Service, Department of Labor. 



The proposed performance measures also improve the way VETS 
establishes the level of performance that states are expected to achieve. 
VETS no longer requires states to compare the level of services provided 
to veterans with those provided to nonveterans . 6 In past reports, we have 
pointed out that the use of these relative standards results in states with 



6 While states will no longer be required to compare the level of services given to veterans 
and nonveterans, VETS is required to report annually to the Congress on the job placement 
rate of veterans compared with the rate for nonveterans. 38 U.S.C. § 4107. 
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poor levels of service to nonveterans being held to lower standards for 
service to veterans than states with better overall performance. 7 For 
example, in program year 1999, Rhode Island reported an entered- 
employment rate of 5.49 percent for nonveterans. Because VETS requires 
states to ensure that they achieve an entered-employment rate for veterans 
that is 15 percent higher than that for nonveterans, Rhode Island’s 1999 
expected performance level was 6.32 percent of registered veterans 
entering employment-a low level of performance. Under the proposed 
system, VETS will negotiate performance levels annually with each state 
based on that state’s past performance, using guidelines similar to those 
used for WIA 8 VETS will also be able to ac^just these levels based on 
economic conditions within each state, such as the unemployment rate, 
the rate of job creation or loss, or other factors. 9 

I 

The proposed performance measures are also similar to those established 
under WIA, making it easier for service providers to achieve WIA’s goal of 
integrating and streamlining employment and training services. In the 
current environment, many of the programs that provide services through 
the one-stop centers have their own unique performance measures and 
program definitions, requiring multiple systems and multiple data 
collection efforts to track a single client. In the proposed system, VETS 
has made an effort to align its performance measures with those of WIA. In 
fact, two of the five proposed measures — entered-employment rate and 
employment retention — are nearly identical to WIA’s and to those 
proposed for ES. If VETS aligns the measures with those of WIA and ES, 
local offices will be more readily able to establish integrated data systems 
that will minimize the data collection burden on service providers and 
clients. (See app. I for a comparison of the WIA performance measures 
with those proposed for VETS and ES.) 



7 See GAO/T-HEHS-97-129. 

8 VETS is planning to use WIA’s negotiation process to establish expected performance 
levels for labor exchange services. VETS proposes that states use 2 years of data if 
possible, but not less than 1 year in determining trends for performance and factors that 
may influence performance. 

9 VETS’ proposal mentions the following as possible factors for states to consider when 
negotiating expected levels of performance: the unemployment rate, the rate of job 
creation/loss and new business start-ups, availability of transportation and daycare, pursuit 
of new or enhanced employment partnerships, natural disasters, and state legislation. 
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VETS Should Address 
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While the proposed performance measures are an improvement over those 
currently in place, there are issues with these measures that VETS should 
address. First, a comparison of the performance measures with the 
strategic plan indicates that VETS is sending a mixed message to states 
about what services to provide and to whom. The strategic plan suggests 
that states focus their efforts on providing staff-assisted services to 
veterans, including case management. Yet, none of the proposed measures 
specifically gauges whether more staff-intensive services are helping 
veterans get jobs. VETS’ proposal includes a measure that tracks 
employment outcomes following staff-assisted services. However, this , 
measure is broadly defined, and the list of staff-assisted services includes 
nearly all services provided to veterans. 10 This makes the outcomes 
achieved for the staff-assisted measure nearly identical to those reported 
for the more general “entered-employment rate.” In addition, as VETS has 
defined it, staff-assisted services include many services that might not be 
considered “intensive,” such as referral to a job and job search activities. 
Because the definition is so broadly defined, a veteran who only attended 
a job search workshop would be counted the same as a veteran who 
received more intensive services, such as testing and employability 
planning. Both would be counted in the more general entered-employment 
rate measure, as well as the staff-assisted service measure. A stricter 
definition for staff-assisted services that includes only those services that 
are generally considered staff-intensive would allow VETS to more 
accurately assess the success of those services and help to clarify the 
goals of the program. 

Second, VETS is sending a mixed message about which groups of veterans 
to target for services. As we noted in past reports and testimonies, VETS 
has inconsistently identified various “targeted” groups of veterans it plans 
to help." In its strategic plan, VETS identifies two broad veteran groups 
that should be targeted to receive special attention — (1) disabled veterans 



10 VETS uses the ETA definition of staff-assisted services. Staff-assisted services include 
(a) referral to a job; (b) placement in training; (c) assessment services, including an 
assessment interview, testing, counseling, and employability planning; (d) career guidance; 
(e) job search activities, including resume assistance, job search workshops, job finding 
clubs, providing specific labor market information and job search planning; (f) federal 
bonding program; (g) job development contacts; (h) tax credit eligibility determination; (i) 
referral to other services, including skills training, educational services, and supportive 
services; and (j) any other service requiring expenditure of time. Application taking and/or 
registration services are not included as staff-assisted services. 



11 See GAO/T-HEHS-00-206. 
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and (2) all veterans and other eligible persons. And consistent with this, 
VETS proposes that expected performance levels be negotiated separately 
for each of these same two groups. Yet, the strategic plan also suggests 
that, when providing services to all veterans, special attention should be 
given to meeting the needs of certain other target groups, some of which 
might require more intensive services to become employed. The groups 
targeted for special attention include (1) veterans who have significant 
barriers to employment, (2) veterans who served on active duty during a 
war (or campaign or expedition in which a campaign badge has been 
authorized), and (3) veterans recently separated from military service. In 
reviewing VETS’ proposed measures and the plan for negotiating 
performance levels, staff may be confused as to where they should place 
their service priorities. It is unclear what steps VETS will take to ensure 
that DVOPS and LVERs are provided ample opportunity and 
encouragement to focus attention on the portion of the “all veterans” 
group who may require more staff time to be successful in getting a job. 

Last, VETS’ proposal also continues to include a performance measure 
related to federal contractor job openings listed with the state’s ES office. 
However, in its proposal, VETS has changed the measure. Under the 
current system, VETS tracks the number of Vietnam-era and special 
disabled veterans who were placed in jobs listed by federal contractors — 
an outcome measure. Now, under the proposed system, VETS will track 
the increase in the number of federal contractor jobs listed with the state’s 
ES office — a process-oriented measure. 12 This new measure ultimately 
holds DVOPS and LVERs accountable for the number of federal 
contractors in a given state or local area, not for veteran placements with 
those contractors. The presence of federal contractors in a given state or 
local area is unpredictable and is determined by the federal agencies 
awarding contracts. Furthermore, according to state officials that we 
talked with, the federal contractor measure should be eliminated 
altogether because it is the responsibility of contractors to list their job 
openings. In addition, it is the Office of Federal Contract Compliance that 
is responsible for ensuring that all companies conducting business with 



12 Any contractor or subcontractor with a contract of $25,000 or more with the federal 
government must take affirmative action to hire and promote qualified special disabled 
veterans, veterans of the Vietnam-era, or any other veterans who served on active duty 
during a war or in a campaign or expedition for which a campaign badge has been 
authorized. Contractors and subcontractors with job openings, other than executive or top 
management jobs, must list them with the nearest state employment office. Veterans cited 
above receive priority for referral to federal contractor job openings listed at those offices. 
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the federal government list their jobs with state ES offices and take 
affirmative action to hire qualified veterans. 



Proposed Bata Source 
for New Measures Is 
An Improvement But 
Will Bring Some 
Challenges 



The proposed data for the new measures will greatly improve the 
comparability and reliability of these measures, but this change will bring 
some challenges that VETS will need to address. Consistent with WIA and 
ES, VETS is proposing that all states use UI wage records to identify 
veterans who get jobs. UI wage records contain the earnings of each 
employee reported quarterly by employers to state UI agencies. 13 
Currently, the data VETS uses are not comparable across states, in part, 
because states use different, data sources to report employment-related 
outcomes. Using a single, standardized source ,for collecting data will 
improve VETS’ ability to compare performance across states. UI wage 
records will also provide state officials with a better means to identify 
veterans who get jobs than does the traditional follow-up method of 
telephoning veterans and/or employers to verify employment. However, 
states cannot readily access wage records from other states, wage records 
do not cover certain types of employment, and these data are not available 
until 3 to 9 months after an individual gets a job. 



Proposed Data Source Will Using a single data source will help to standardize the way in which states 
Help To Ensure collect data on veterans, thereby making it easier to compare performance 

Comparability and across states. Currently, states are using various data sources for 

Reliability Across States performance-reporting purposes. While almost all of the states in our 

review used a combination of data sources to determine whether or not a 
veteran got a job, most of the states relied substantially on one data 
source, but that source differed among states. For example, in program 
year 1999 

• 7 of the 15 states that we contacted relied to a large extent on wage 
record data to determine whether a veteran got a job or not; 

• 7 others relied, for the most part, on telephone calls and letters to 
veterans and employers to determine a veteran’s employment status; 
and 



13 Each calendar quarter, employers in a state provide wage information on their employees 
to their state’s UI agency or some other state agency. The information contained in wage 
records varies from state to state. However, all wage records contain at least the following 
information: the calendar quarter that the wages were reported in, the employee’s social 
security number, wages paid to the employee in that quarter, and employer information. 
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• one state relied primarily on its new hire database for employment 
data. 14 

In addition to making state data more comparable, we found evidence that 
states currently using wage records have been able to better identify those; 
veterans who get jobs after receiving services. A recent study found that 
UI wage records more accurately identified how many veterans got, jobs , 
after receiving DVOP, LVER, or ES services. 16 Using UI wage records, this ; 
study tracked veterans who registered with the Maryland Job Service 
during program year 1997 and found an entered-employment rate that 
ranged from 65 percent to 82 percent, depending on the way the study 
defined a registrant. In that same program year, Maryland reported to 
VETS an entered-employment rate of 31 percent, which was based on staff 
telephoning veterans and employers to verify employment. In addition' 
most states in our review that are now using UI wage records, either as 
their primary data source or to augment other data sources, reported 
higher employment rates in program year 1999 for veterans they served 
than that year’s national average of 30 percent. (See app. 11 for a list of all 
states and their respective entered-employment rates for program years 
1996-1999.) By comparison, all but one of the states that relied either on 
manual follow-up or the new hire database reported an employment rate 
below the national average. 

Another benefit of using UI wage records is that staff assisting veterans 
will be relying on data already available rather than collecting additional 
information from veterans or employers. Relying on these already 
reported data would require less staff time from DVOP, LVER, and ES 
staff, freeing them to focus more on providing job-related services to 
veterans. State officials told us that relying on manual follow-up, such as 



14 The Personal Responsibility and Work Opportunity Reconciliation Act of 1996 
established the National Directory of New Hires and State Directories of New Hires. The 
National Directory is maintained by the Social Security Administration on behalf of the 
Department of Health and Human Services, Office of Child Support Enforcement. States 
maintain their own State Directories of New Hires and generally supply data for the 
National Directory. Information in these directories includes: new hire information, such as 
name, address, and social security number of the employee and the name, address, federal 
identification number of the employer; in some states, wage information; and UI claim 
information. 

16 Proposed New Entered Employment Patterns of Veteran Wagner-Peyser Registrants in 
the State of Maryland, by Robert Cook, BETAH Associates; and Edward Davin and Karin 
Willner, DynCorp (Apr. 12, 2000). 
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telephone calls, has been labor-intensive and has diverted staff attention 
away from providing appropriate assistance to veterans. 16 


UI Data Presents Some 
Challenges 


While UI wage records offer advantages over the current data collection 
system, some challenges need to be addressed. First, states should find 
ways to identify interstate job placements. Because the UI wage record 
system resides within each state, states generally do not have access to 
wage records from other states, making it difficult to track individuals who 
receive services in one state but get a job in another. Currently, there is no 
national system in place that facilitates data sharing among states. 
However, in response to WIA requirements, states are developing an 
interstate UI wage record information sharing system, known as the Wage 
Record Interchange System (WRIS). The system is designed to minimize 
the burden on state unemployment insurance programs in responding to 
requests for wage record data, to ensure the security of the transactions 
involving individual wage records, and to produce the results at a low cost 
per record. In addition, some states have entered into agreements with 
neighboring states to share wage information in support of WIA. These 
efforts should help VETS as well. 

Second, states should find ways to identify those veterans finding jobs in 
categories not covered by UI wage records. UI wage records cover about 
94 percent of wage and salary workers, but certain employment categories 
are not covered, such as self-employed persons, most independent 
contractors, military personnel, federal government workers, railroad 
employees, some part-time employees of nonprofit institutions, and 
employees of religious orders. Therefore, the UI system will not be able to 
track and count veterans who get these types of jobs. This is an issue for 
WIA as well, and states are beginning to assess the extent to which this 
issue will affect their ability to accurately determine the outcome of WIA- 
funded programs. 
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16 See draft report, “Measuring Employment and Income for Low-Income Populations with 
Administrative and Survey Data,” V. Joseph Holtz, University of California at Los Angeles 
and John Karl Scholz, University of Wisconsin, June 30, 2000, for a discussion of the 
benefits of UI data compared to contacting program participants for performance 
reporting. 
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Other Measurement Issues 
Affect Comparability of 
States’ Performance Data 



There are other issues not related to the use of UI wage records that VETS 
should consider as it finalizes its performance-reporting requirements. 
VETS’ proposed performance system does not standardize how states 
report veterans or nonveterans who use self-service activities, making it 
difficult to reliably assess nationwide performance. In an environment in ; 
which self-service is becoming more common, we found that states vary in/, 
whether they register veteran job seekers who access self-service tools, 
such as internet-based job listings or resume writing software. For 
example, some states allow job seekers greater access to job listings 
without requiring that they register, while others have more restrictions on 
who can access job lists. Table 3 shows how such differences can affect 
entered-employment rates. In this example, 100 veterans enter the 
employment service for assistance. In both cases, 40 veterans ultimately 
get jobs after receiving identical services. In one case, the placement rate 
is 40 percent and in the other, 50 percent — a 10 percentage point 
difference. This difference results from counting all job seekers in one 
case and only those requiring staff assistance in the other. As a result of _ 
the different ways states currently count veterans and report outcomes, 
the entered-employment rate measure is not consistently calculated across 
states, and nationwide comparisons are misleading. 



Table 3: A Comparison of Entered-Employment Rates by Registration Policy 



All veterans required to register 


Veterans accessing self-service do not have to register 




Veterans 

registered 


Number of 
veterans 
who get 
jobs 


Number of 
veterans with 
jobs counted 
in entered- 
employment 
rate 




Veterans 

registered 


Number of 
veterans who 
get jobs 


Number of 
veterans with 
jobs counted in 
entered- 
employment rate 


40 Veterans 
use self- 


40 


10 


10 


40 Veterans 
use self- 


0 


10 


0 


service 








service 








60 Veterans 


60 


30 


30 


60 Veterans 


60 


30 


30 


require staff 
assistance 








require staff 
assistance 








Total 


100 


40 


40 


Total 


60 


40 


30 


Reported Entered-Employment Rate: 40/100 


= 40% 


Reported Entered-Employment Rate: 30/60 = 


50% 



Source: GAO analysis. 




VETS’ proposed performance system does not standardize how long a 
veteran or nonveteran remains registered after seeking services for 
performance-reporting purposes. We found that states differ in how long 
they keep veterans registered. This difference affects the calculation of the 
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Conclusions 
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entered-employment rate (i.e., the number of veterans that get jobs), 
making performance comparisons across states less reliable. Many of the 
states we contacted count individuals as registered who have received a 
service in the last 6 months. However, two states only count those as 
registered who have received a service in the last 3 months, while two 
others count only those who received a service in the last 2 months. And in 
one state, anyone who has received a service from the state’s employment 
office since 1998 is counted as a registrant when determining the entered- 
employment rate. States with shorter registration periods may be able to 
report a higher entered-employment rate than states with longer 
registration periods. 



VETS is improving its performance measurement system by proposing 
new measures that are more outcome-oriented than its current measures 
and by requiring that all states use wage record data to improve the 
comparability and reliability of reported program performance. While 
these changes move VETS a step closer to implementing an effective 
accountability system, they may not go far enough. VETS continues to 
send a mixed message to states about what services to provide and to 
whom. As presently defined, two of the proposed measures — the entered- 
employment rate and the employment rate following staff-assisted 
services — may provide nearly identical results, and neither helps VETS to 
monitor whether more intensive services are being provided to veterans or 
whether these services are successful. VETS also continues to 
inconsistently identify the groups of veterans that it wants states to help. 

In addition, VETS maintains a measure related to federal contractors — one 
that is beyond the control of DVOPS and LVERs. 

Furthermore, in its proposed system, VETS allows states to decide which 
veterans to include in its performance reports. This results in data 
inconsistencies that make state-to-state comparisons unreliable. Without 
clear and consistent direction from VETS’ planning documents and 
performance measures, staff assisting veterans will be uncertain where to 
place their priorities. In addition, without stricter guidelines for how to 
count veterans, VETS will be unable to accurately assess program 
performance nationwide. Unless further modifications are made, VETS 
will be unable to fully determine whether its programs and services are 
fulfilling its mission. 
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In order to establish a more effective performance management system, 
we recommend that the Secretary of Labor direct VETS to do the 
following: 

• Redefine staff-assisted services to include only those that may be 

considered staff intensive, such as case management, so that VETS will be \ 
able to evaluate the success of intensive staff-assisted services. t 

• Clearly define target populations so that staff assisting veterans know 
where to place their priorities. If staff are to focus on assisting veterans 
who need more assistance, VETS should provide incentives and 
opportunities to do so through appropriate performance measures or 
negotiated levels of performance. 

• Eliminate the measure related to federal contractor jobs so that staff are 
not held accountable for the number of federal contractor^ in a state oi - 
local area or for the failure of contractors to list their jobs with ES offices. 

• Establish and communicate guidelines that standardize how to count 
veterans for performance-reporting purposes so that VETS will be able to 
assess program performance nationwide. 



We provided VETS with the opportunity to comment on a draft of this 
report. Formal comments from VETS appear in appendix III. In addition 
to the comments discussed below, VETS provided technical comments 
that we incorporated where appropriate. 

VETS generally agreed with our findings and two of our recommendations 
but disagreed with the other two recommendations. VETS acknowledged 
that its current strategic plan (Nov. 2000) sends a mixed message to the 
states about which groups of veterans staff should target for special 
attention. VETS noted that it is revising its strategic and annual plans to 
reflect a more consistent message about what services to provide and to 
whom. VETS also explained that it is developing new performance 
standards specific to DVOP and LVER staff that will clarify the role they 
play in providing services to veterans. According to VETS officials, states 
will have the option of using these specific standards or developing their 
own. When developing these standards, VETS will need to ensure that the 
specific standards developed for DVOPS and LVERs are consistent with 
the message in the revised strategic plan and that together they provide a 
coherent strategy as to where staff should place their service priorities. 

VETS disagreed with our recommendation for a revised definition of the 
performance measure related to staff-assisted services. VETS said that any 
veteran receiving staff-assisted services may require a multitude of the 
services cited in the definition — any one of which or combination thereof 
may require extensive staff time. We disagree that any one of these 
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services necessarily requires extensive staff time. As noted in our report, a 
veteran may be counted as receiving staff-assisted services after receiving 
only a job referral or labor market information — services that by 
themselves would not involve extensive staff resources. Moreover, we 
continue to believe that the broadly defined staff-assisted service measure 
will likely not report outcomes substantially different from those reported 
for the more general entered-employment rate measure. As noted in our 
report, a stricter definition for staff-assisted services that includes only 
those services generally considered to be staff-intensive would allow VETS 
to more accurately assess outcomes associated with those services. 

i 

VETS disagreed with our recommendation to discontinue the measure 
related to jobs listed by federal contractors. However, VETS agreed to 
reconsider the suitability of this specific measure after public comments 
have been received. As we noted in our report, the presence of federal 
contractors in a given state or local area is determined by the federal 
agencies awarding contracts. In addition, state officials told us that it is 
the responsibility of the contractors, not DVOP and LVER staff, to list their 
job openings with employment services. Current law requires the 
Secretary of Labor to report annually to the Congress on the number of 
federal contractor positions listed and the number of veterans receiving 
job priority through this program. This information could be collected in 
absence of a specific performance measure. 

With regard to our recommendation that VETS establish guidelines that 
standardize how states count veterans for performance-reporting 
purposes, VETS said that it will be working with ETA to determine how 
states can uniformly report veterans and nonveterans that use self-service 
activities. In addition, VETS noted that the revised ETA 9002 report will 
provide uniform instructions on how long individuals remain registered in 
the system. 



We are sending copies of this report to the Honorable Elaine L. Chao, 
Secretary of Labor; appropriate congressional committees; and other 
interested parties. We will also make copies available to others upon 
request. 
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If you or your staff have questions about this report, please contact me on 
(202) 512-7215 or'Dianne Blank on (202) 512-5654. Individuals making key 
contributions to this report include Elizabeth Morrison and Amanda 
Ahlstrand. 



Sincerely yours, 




Sigurd R. Nilsen, Director 
Education, Workforce and 
Income Security Issues 
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Appendix I: Comparison of VETS, ES, and 
WIA Performance Measures 

i 1 


Similar to Workforce Investment Act (WIA) programs, the Employment 
Service (ES) and the Veterans’ Employment and Training Service (VETS) 
are proposing that their programs use Unemployment Insurance wage 
records to report on performance measures. Each calendar quarter, 

, . , employers submit wage record data to their state’s UI agency or some 

other state agency. The following table compares the proposed 
performance measures of VETS and ES and those used by WIA’s adult and 
dislocated worker programs. ' 


' 


VETS proposed performance 
measures 


ES proposed performance 
measures 


WIA performance measures (adult and 
dislocated worker programs) 


Entered-employment rate: 

The percentage of all registered 
veterans who got a job in the 1st or 

2nd quarter after registration. 


Entered-employment rate: 

The percentage of workers who got a job 
in the 1 st or 2nd quarter after registration. 


Entered-employment rate: 

The percentage of workers who got a job 
by the end of the 1st quarter after exit. 


Employment retention rate at 6 
months: 

Of the veterans who got a job after 
registration, the percentage who 
were still earning wages in the 2 nd 
quarter after getting a job. 


Employment retention rate at 6 
months: 

The percentage of workers who 
continued to earn wages in the 2 nd 
quarter after the 1st quarter in which 
there were earned wages. 


Employment retention rate: 

Of those who had a job in the 1st quarter 
after exit, the percentage of workers who 
have a job in the 3rd quarter after exit. 


Employment rate following receipt 
of staff-assisted services: 

Of the veterans who received staff- 
assisted services, the percentage 
who got a job in the 1 st or 2nd 
quarter after registration. 0 


No measure 


No measure 


Federal contractor job openings 
listed with the public labor 
exchange: 

The percentage increase in the 
number of federal contractor Job 
openings listed annually with the 
public labor exchange from one 
program year to the next. 


No measure 


No measure 


No measure 


Employer customer satisfaction: 
Average of three survey questions on 
employers’ satisfaction with services 
received. 


Employer customer satisfaction: 
Average of three survey questions on 
employers’ satisfaction with services 
received. 


No measure 


Job seeker customer satisfaction: 
Average of three survey questions on job 
seekers’ satisfaction with services 
received. 


Job seeker customer satisfaction: 
Average of three survey questions on job 
seekers’ satisfaction with services 
received. 
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WIA Performance Measures 

i 

i 

* i 


VETS proposed performance 
measures > 


ES proposed performance 
measures 


WIA performance measures (adult and 
dislocated worker programs) 


No measure j 

\ 


No measure 


Earnings change (adults only): 

The difference between total post-program 
earnings (from the 2nd and 3rd quarters 
after exiting the WIA program) andjthe ; 

total pre-program earnings (from the 2nd 
and 3rd quarters prior to entering the WIA | 
program) divided by the number of ( 

participants leaving the program. ' 


No measure 


No measure 


Earnings replacement rate (dislocated / 
workers only): 1 

Total post-program earnings (in the 2nd 
and 3rd quarters after exit) divided by pre- 
dislocation earnings (in the 2nd and 3rd 
quarters prior to dislocation). 



“Staff-assisted services include: (a) referral to a job; (b) placement in training; -(c) assessment 
services, including an assessment interview, testing, counseling and employability planning; (d) 
career guidance; (e) job search activities, including resume assistance, job search workshops, job 
finding clubs, specific labor market information and job search planning; (f) federal bonding program; 
(g) job development contacts; (h) tax credit eligibility determination; (i) referral to other services, 
including skills training, educational services and supportive services; and (j) any other service 
requiring expenditure of time. Application taking and/or registration services are not included as staff- 
assisted services. 

Source: Veterans’ Employment and Training Service, U.S. Department of Labor; U.S. Department of 
Labor Training and Employment Information Notice Number 13-000, “Consultation Paper on Labor 
Exchange Performance Measurement System;” and U.S. Department of Labor Training and 
Employment Guidance Letter Number 7-99, Xtore and Customer Satisfaction Performance Measures 
for the Workforce Investment System.” 




Page 23 



26 



GAO-0 1-15 80 VETS’ Performance Measures and Data 



Appendix II: States’ Entered-Employment 
Rates for Veterans in Program Year 1999 



i 



Entered-employment rate 



State 


1999 


1998 


1997 


1996 


Alabama 


38.4 


40.4 


38.9 


34.3 


Alaska 


22.6 


20.8 


25.8 


22.9 


Arizona 


32.4 


29.4 


21.4 


18.9 


Arkansas 


30.3 


29.5 


25.6 


31.8 


California 


5.2 


3.7 


15.0 


18.5 


Colorado 


30.4 


28.7 


27.9 


26.1 


Connecticut 


27.4 


23.7 


21.2 


22.2 


Delaware 


17.1 


12.1 


11.8 


12.8 


District of Columbia 8 


20.3 


17.0 


15.1 


9.5 


Florida 


24.5 


19.4 


21.2 


20.5 


Georgia 


, 38.1 


34.5 


30.0 


26.1 


Hawaii 


22.2 


15.5 


16.8 


14.5 


Idaho 


30.5 


30.0 


29.0 


30.3 


Illinois 


35.8 


34.5 


30.4 


28.9 


Indiana 


21.9 


14.1 


18.8 


16.7 


Iowa 


46.1 


48.6 


44.4 


45.2 


Kansas 


23.3 


23.7 


26.6 


23.8 


Kentucky 


26.7 


28.7 


24.8 


25.0 


Louisiana 


29.3 


15.5 


18.3 


16.1 


Maine 


22.7 


24.0 


22.6 


13.6 


Maryland 


34.0 


31.1 


27.6 


25.8 


Massachusetts 


42.2 


35.5 


31.2 


25.2 


Michigan 


17.0 


18.4 


6.8 


8.7 


Minnesota 


32.9 


18.4 


20.2 


20.9 


Mississippi 


33.3 


31.8 


30.7 


30.5 


Missouri 


33.9 


24.9 


32.5 


30.9 


Montana 


33.1 


30.2 


29.2 


31.7 


Nebraska 


25.5 


24.1 


26.5 


26.3 


Nevada 


26.9 


24.2 


27.6 


28.5 


New Hampshire 


37.1 


36.6 


27.4 


23.6 


New Jersey 


34.4 


35.6 


39.9 


40.0 


New Mexico 


30.9 


30.5 


29.8 


17.7 


New York 


20.5 


20.7 


19.7 


18.2 


North Carolina 


44.7 


44.7 


38.7 


38.5 


North Dakota 


50.9 


48.5 


47.5 


38.8 


Ohio 


18.5 


16.1 


18.5 


15.8 


Oklahoma 


45.7 


41.8 


44.1 


44.2 


Oregon 


36.2 


33.8 


28.9 


33.0 


Pennsylvania 


33.6 


26.2 


23.0 


21.8 


Puerto Rico a 


13.8 


17.2 


15.6 


18.1 


Rhode Island 


15.6 


12.1 


7.3 


8.9 


South Carolina 


36.8 


35.7 


32.8 


30.7 


South Dakota 


61.3 


58.1 


44.2 


40.6 


Tennessee 


54.4 


68.9 


47.2 


20.3 
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Appendix II: States' Entered- 
Employment Rates for Veterans 
in Program Year 1999 
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Entered-employment rate 



State 


1999 


1998 


1997 


1996 


Texas 


45.8 


36.1 


38.0 


35.3 


Utah 


45.7 


33.5 


41.7 


45.8 


Vermont 


25.3 


17.6 


18.4 


18.0 


Virgin Islands 8 


30.3 


23.0 


15.3 


17.6 


Virginia 


32.3 


23.2 


18.8 


14.1 


Washington 


17.4 


20.8 


25.0 


24.7 1 j 


West Virginia 1 


16.4 


15.4 


13.8 


151 /’ 


Wisconsin 


42.6 


43.8 


44.4 


10.9 / 


Wyoming 


32.3 


29.0 


28.8 


28.3 



“Shown as states for this report. 

Source: Prepared by GAO from data provided by the Veterans’ Employment and Training Service for 
program years 1996 through1999. 
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Appendix III: Comments From the 
Department of Labor 



l 



U.S. Department of Labor Assistant Secretary for 

Veterans Employment and Training 
Washington, DC 20210 



April 30, 2001 



Mr. Sigurd R. Nilsen 
Director 

Education, Workforce, and 
Income Security Issues 
U.S. Government Accounting Office 
Washington, DC 20548 



Dear Mr. Nilsen: 

This letter is provided in response to the draft report GAO-0 1 -580, entitled Veterans' 
Employment and Training Service. Proposed Perfonnaoce Measuremen t System Imp roved. 
But Further Changes Needed. Thank you for providing the Department of Labor and 
Veterans’ Enq>k>yment and Training Service (VETS) the opportunity to comment on the 
draft report. 

VETS recognizes the contributions that the GAO has made during VETS’ redesign of the 
proposed veterans’ performance measures for the public employment service program. Our 
process started with the 1997 General Accounting Office’s report - Focusing on Program 
Results to Improve Agency Performance . One of the steps VETS took in response to the 
GAO’s concerns about the current performance measures was to have several States 
develop and test alternate performance measures. Using the information gained from the 
pilot states, VETS researched the use of the Unemployment Insurance (UI) wage records as 
a means of verifying performance outcomes for veterans entering employment. 

The United States Employment Service (USES) and VETS worked together to redefine the 
proposed change to the current data collection system emitted “ETA 9002 Report.” 

Further, VETS revised its VETS-200 information collection to mirror the changes made to 
the ETA 9002. Until the data elements and new performance measures were agreed upon, 
VETS decided not to modify its Strategic or Annual Performance plans. Recently, VETS 
and USES have completed the revisions to the ETA 9002 and VETS-200. Consequently, 
VETS changed its FY 2002 Annual Performance Pten to reflect new goals, new strategies 
and the new data elements. VETS will be using this plan as a transition vehicle. The 
Strategic Plan will also be revised over the next several months and sho uld address many of 
the GAO Report’s comments. 
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Current law, at Chapters 4 1 and 42 of title 38, United States Code, requires the entire public 
employment service program to provide maximum employment and tra inin g opportunities to 
veterans and other eligible persons. Further, the law provides for Disabled Veterans 
Outreach Program (DVOP) and Local Veterans Employment Representative (LVER) staff 
to enhance services to veterans. The key component of this law is and has been the 
recognition that it is the entire public employment service program that must provide 
maximum service to veterans and otter eligible persons. 

Although VETS agrees with most of the GAO recommendations, there are several areas 
that need closer examination. One is the recommendation to redefine the “Entered 
Employment Rate Following Staff-Assisted Services” measure. Because it is the entire 
public employment service system that is being measured, it is crucial that the outcome 
measurement "Entered Employment Rate Following Staff-Assisted Services" is retained as 
is, as the measure used to determine the success of the public employment service in 
maximizing services to veterans. 

Further, as the GAO report notes, “VETS* [DOL’s] proposed performance system does not 
standardize how states report veterans [or non-veterans] who use self-service activities, 
making it difficult to reliably assess nationwide performance.” This is a Departmental 
concern. ETA and VETS will be working on this issue. 

The entered employment rate following staff-assisted services measure, combined with data i 
from otter reports, identifies services provided by the public employment service and allows 
VETS to identify who is providing services to veterans. Also, States must register all 
applicants provided staff-assisted services. Thus, VETS is seeking uniformity in registration 
among States by using this measure. 

Another area of concern is GAO’s comment on the federal contractor program. Recently 
enacted legislation reflects a policy emphasis on the federal contractor program (38 U.S.C. 

§ 4212). In response to this emphasis, VETS developed new took to identify federal 
contractors as well as the means to measure increases in job listings by these employers as 
well as their subcontractors. VETS recognizes GAO’s concern about this measurement. 

Thus, VETS will reconsider the suitability of this proposed measure after public comment. 

VETS is not, however, inclined to drop a federal contractor measurement of success at this 
time. 

Following are our comments and suggestions, keyed to the appropriate draft report page: 



Page 4: Veterans’ Programs 

VETS is concerned that the report indicates that both DVOP and LVER staff provide the 
same services. VETS is continually working with States to ensure they recognize the 
different statutory responsibilities of DVOP and LVER staff The statute provides different 
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Now on p. 12. 





rotes and clearly outlines different duties for DVOP and LVER staff! (38 U.S.C. §§ 4 103 A 
and 4104). VETS will be taking a more aggressive posture to clarify this difference in its 
strategic and annual performance plans, and its guidance to the States. VETS would 
appreciate the GAO's assistance in stressing the difference in the rotes of DVOP and LVER 
staff 

For example, a key statutory responsibility of LVERs is to provide program oversight of 
local employment service offices to ensure veterans receive maximum employment and 
training opportunities from the entire local office staff (38 U.S.C. 4104(bXl)). For 
DVOPs, the key responsibility is “(1) Development of job and job training opportunities for 
such veterans through contacts with employers, especially small and medium-size private 
sector employers.” (38 U.S.C. 4l03A(e)). 

Page 6: Legislative and Regulatory Changes Affecting VETS 

VETS suggests that the following be added at the end of the first paragraph; 

VETS has implemented the requirements of title 38 of the United States Code through 
agreements with each state on planned services to veterans to include provisions on 
how DVOPS and LVERS will be integrated into the one-stop delivery system. 

Page 11: Concerns Remain That VETS Should Address 

VETS agrees that it sent a mixed message to the States through its previous strategic and 
annual plans. Based on the changing environment and comments from stakeholders, 
including the GAO, VETS is redefining its message by revising its strategic and annual 
plans. 

(See also response to the targeted veterans issues under Page 12). 

VETS' intent is to encourage the public employment service program to provide staff- 
assisted services to veterans based on their needs, white also providing access to self-service 
computers by job ready veterans. This measure is the only current means available to 
determine the success of staff-assisted services provided by the public employment service 
program. Because staff-assisted services require applicant registration, the Entered 
Employment Rate Following Receipt of Staff- Assisted Services” is presently the only 
uniform outcome measure among States. 

VETS does not concur with the GAO’s opinion that the definition of staff-assisted services 
is too broad. Any veteran applicant receiving staff-assisted services may require a multitude 
of these services any one of which, or combination thereof may require extensive staff time. 
For example, a referral to a job may require skills assessment, career guidance, employer 
contact and follow-up. 
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The public employment service’s mission is to provide employment services to employers 
and job applicants. With declining staff resources the public employment service cannot 
provide extensive services to its customers. Applicants requiring “intensive services” as 
defined by WI A, are referred to WIA service providers by the public employment service 
staff. The WIA “intensive services” by definition, are those set aside for specific WIA 
service operators or through contracts, and include for example, “comprehensive and 
specialized assessments of the skill levels and service needs of adults and dislocated workers 
. . . “individual counseling and career planning”; and, “case management for participants 

seeking training services “ Thus, VETS has focused DVOP services to address the 

needs of those veterans, particularly disabled veterans, who would benefit from case 
management and employment development activities. VETS is devetoping new prototype 
standards of performance for DVOPs and LVERs and related reporting requirements. 

VETS will introduce these standards in FY 2002. 



Page 12: Concerns Remain That VETS Should Address 

VETS acknowledges that its current strategic plan may not have been clear on the other 
target groups served by the public employment service. However, the other groups served 
because of other statutory responsibilities will be measured separately, and will not be 
brought into this proposal for measuring services. The other targeted veterans are better 
served by the Veterans Workforce Investment Programs, Homeless Veterans Reintegration 
Projects, and Transition Assistance programs as well as other initiatives and should be 
measured separately from this proposal VETS will also clarify its position that the public 
employment service must provide staff-assisted services to veteran job seekers and the need 
to measure the result of these services. This will enable DVOP and L VER staff to focus 
their attention on their statutory responsibilities. VETS' development of new prototype 
standards will clarify the role of the DVOPs and LVERs. 

The inclusion of the federal contractor performance measure is related to several factors, the 
most important of which is the recognition that Congress intends that the Department 
devote more attention and place more emphasis on the federal contractor program as 
reflected by public laws amending 38 U.S.C. § 4212 during the 105* and 106* Congresses 
(Public Laws 105*339 and 106-416). 

Additionally, the proposed measure shows that VETS recognizes the importance of 
employers to the labor exchange system. Not only do employers fund the trust account that 
supports the public employment service, including VETS, but also employer listings are 
necessary for the operation of a labor exchange system. Employers, in turn, benefit from 
having qualified applicants to choose from when filling their vacancies. VETS recognizes 
that increasing the number of job listings by employers, particularly those listed by federal 
contractors and their subcontractors, as required of them by 38 U.S.C. § 4212, is a benefit 
not only for veteran job seekers, but also for employers with federal contracts or 
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subcontracts. This proposed performance measure is being published for comment, and 
VETS will reconsider its suitability after review of the comments received. 

The public employment service, not DVOP and LVER staf£ is responsible at the local level 
for encouraging federal contractors to list their vacancies, and to make priority referral of 
veterans to such openings of qualified veterans. 



Page 17: Other Measurement Issues Affect Comparability of States* Performance Data 

The hew ETA 9002 specifications provide uniform instructions on how long individuals 
remain registered in the system. It must be noted that WI A enables States to determine 
when an individual is registered, by yielding to State law or policy except when staff-assisted 
services are provided. 

In conclusion, VETS truly appreciates the GAO’s effort to assist VETS in devetoping 
meaningful performance measures. VETS will be glad to answer any questions that may 
arise. I can be reached 202-693-4700. 
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